A High Performance Redundancy Scheme for Cluster File Systems

A known problem in the design of striped file systems is their vulnerability to disk failures. In this paper we address the challenges of augmenting an existing file system with traditional RAID redundancy, and we propose a novel hybrid redundancy scheme designed to maximize disk throughput as seen by the applications. To demonstrate the hybrid redundancy scheme we build CSAR, a proof-of-concept implementation based on PVFS, and test its performance using both microbenchmarks and representative scientific applications. While either one of traditional schemes such as RAID1 and RAID5 can deliver the highest bandwidth depending on the nature of the load, the hybrid scheme consistently performs as the best of the two. The application-dependent, potentially larger storage occupation of our scheme is justified by current technological trends that put I/O bandwidth at a premium over disk space.